Imaging system with time-of-flight sensing

ABSTRACT

A time-of-flight (TOF) sensing system may include an illumination module and a sensor module. The illumination module may emit light having a reduced illumination duty cycle such as a high peak power and low width illumination pulse. The emitted light may reflect off of one or more objects as reflected light. The sensor module may include pixels operable based on a sensor modulation signal to generated image charge portions in response to the reflected light. The sensor modulation signal may be selectively suppressed after a period of time corresponding to a distance of interest. Processing circuitry in the TOF sensing system may obtain TOF information based on a phase difference between the emitted light and the image charge portions determined by cross-correlation data. By using the illumination pulse and the selective sensor modulation suppression, TOF sensing may exhibit reduced aliasing issues and improved ambient light rejection.

This application claims the benefit of U.S. provisional patentapplication No. 63/015,756, filed on Apr. 27, 2020, which isincorporated by reference herein in its entirety.

BACKGROUND

This relates generally to imaging systems and more specifically tocamera modules having time-of-flight (TOF) sensing capabilities.

A typical TOF sensor includes an illumination module and a sensormodule. The illumination module emits light onto an image scene havingone or more objects. The emitted light reflects off of the one or moreobjects and is received by pixels in the sensor module to generatecorresponding electrical charge. Based on the received light (e.g., thegenerated electrical charge), the sensor module can performtime-of-flight sensing calculations or operations to determine depth andother scene information.

In the illustrative example of indirect TOF sensing, the illuminationmodule can emit light having a specific modulation frequency. Thereflected light, when received by the sensor module, can have the samemodulation frequency with a phase delay (e.g., a time delay) relative tothe emitted light indicative of a distance traveled by the emitted andreflected light. The sensor module can determine the phase delay basedon the generated image data and can therefore determine depth and otherscene information, which are correlated with the determined phase delay.

However, this type of indirect TOF sensing and processing can sufferfrom issues such as aliasing or depth ambiguity issues (e.g., resultingfrom the harmonic or repetitive nature of the emitted light reaching andreflected from different objects in the scene), ambient light saturationissues (e.g., resulting from the continual gathering of ambient lightcharge), and ambient light shot noise issues (e.g., resulting from theinability to remove such noise once ambient light charge has beencollected at the pixel level).

It is within this context that the embodiments herein arise.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an illustrative imaging system having atime-of-flight sensor in accordance with some embodiments.

FIG. 2 is a diagram of illustrative sensor circuitry having a pixelarray and corresponding control and readout circuitry in accordance withsome embodiments.

FIG. 3 is a circuit diagram of an illustrative sensor pixel inaccordance with some embodiments.

FIG. 4 is a diagram of light emitted from an illumination module andlight reflected from one or more objects that is collected by a sensormodule in accordance with some embodiments.

FIG. 5 is a diagram of illustrative illumination characteristics of anillumination module, illustrative sensor modulation characteristics of asensor module, and illustrative cross-correlation characteristics usedto determine time-of-flight information in accordance with someembodiments.

FIG. 6 is a diagram of illustrative illumination characteristics havingperiodic narrow pulses with low duty cycle, illustrative sensormodulation characteristics, and illustrative cross-correlationcharacteristics in accordance with some embodiments.

FIG. 7 is a diagram of illustrative illumination characteristics havinga single high peak power pulse with low duty cycle, illustrative sensormodulation characteristics, and illustrative cross-correlationcharacteristics in accordance with some embodiments.

FIG. 8 is a diagram of illustrative illumination characteristics havinga single high peak power pulse with low duty cycle, illustrative sensormodulation characteristics exhibiting selective sensor modulationsuppression, and illustrative cross-correlation characteristics inaccordance with some embodiments.

DETAILED DESCRIPTION

Electronic systems such as digital cameras, computers, cellulartelephones, automotive systems, and other electronic systems may includeimaging systems or modules that gather light (photons) to capture one ormore images (one or more image frames) that include information abouttheir surrounding environments (e.g., the image scenes). The imagingsystem may have sensor circuitry including one or more arrays of imagesensor pixels (sometimes referred to herein as sensor pixels or pixels).The active pixels in the array may include photosensitive elements suchas pinned photodiodes that convert the incoming light into electriccharge. The array may have any number of pixels (e.g., hundreds orthousands or more). Sensor circuitry may include control circuitry suchas circuitry for controlling the pixels and readout circuitry forreading out image signals corresponding to the electric charge generatedby the photosensitive elements.

FIG. 1 is a diagram of an illustrative imaging system such as anelectronic device that uses sensor circuitry (sometimes referred toherein as a sensor module) to capture images. Imaging system 10 of FIG.1 may be a stand-alone camera, a cellular telephone, a tablet computer,a webcam, a video camera, a video surveillance system, an automotiveimaging system, a video gaming system with imaging capabilities, anaugmented reality and/or virtual reality system, an unmanned aerialvehicle system (e.g., a drone), an industrial system, or any otherdesired imaging system or device that captures image data. Camera module12 (sometimes referred to as an imaging module or image sensor) may beused to convert incoming light into digital image data. Camera module 12may one or more corresponding sensor modules 16. During image captureoperations, light from a scene may be focused onto sensor module 16 byone or more corresponding lenses. Sensor module 16 may include circuitryfor generating analog pixel image signals and circuitry for convertinganalog pixel image signals into corresponding digital image data, asexamples. The digital image data may be provided to storage andprocessing circuitry 18.

Storage and processing circuitry 18 may include one or more integratedcircuits (e.g., image processing circuits, microprocessors, storagedevices such as random-access memory and non-volatile memory, etc.) andmay be implemented using components that are separate from camera module12 and/or that form part of camera module 12. When storage andprocessing circuitry 18 is implemented on different integrated circuitsthan those implementing camera module 12, the integrated circuits withcircuitry 18 may be vertically stacked or packaged with the integratedcircuits for camera module 12. Image data that has been captured bycamera module 12 may be processed and stored using processing circuitry18 (e.g., using an image processing engine on processing circuitry 18,using an imaging mode selection engine on processing circuitry 18,etc.). Processed image data may, if desired, be provided to externalequipment (e.g., a computer, an external display, or other devices)using wired and/or wireless communications paths coupled to processingcircuitry 18.

In some illustrative configurations described herein as examples, cameramodule 12 may implement a time-of-flight (TOF) sensor or camera. Inthese configurations, camera module 12 may include illumination module14 configured to emit light for illuminating an image scene (e.g., oneor more objects in the image scene), and sensor module 16 may beconfigured to gather reflected versions of the emitted light and togenerate TOF information for the image scene (e.g., depth or distanceinformation for one or more of the objects, a depth or distance map ofthe image scene, an image of the image scene, other informationindicative of TOF information, etc.). Additional details forimplementing the specific features of such TOF sensing systems aredescribed below.

As shown in FIG. 2, sensor module 16 (e.g., in TOF sensor 12 in FIG. 1)may include a pixel array 20 containing sensor pixels 22 arranged inrows and columns and control and processing circuitry 24. Array 20 maycontain, for example, tens, hundreds, or thousands of rows and columnsof sensor pixels 22. Control circuitry 24 may be coupled to row controlcircuitry 26 (sometimes referred to as row driver circuitry or pixeldriver circuitry) and column readout and control circuitry 28 (sometimesreferred to as column readout circuitry or column control circuitry,readout circuitry, or column decoder circuitry). Row control circuitry26 may receive row addresses from control circuitry 24 and supplycorresponding row control signals such as reset, anti-blooming, rowselect (or pixel select), modulation, storage, charge transfer, andreadout control signals to pixels 22 over row control paths 30. One ormore conductive lines such as column lines 32 may be coupled to eachcolumn of pixels 22 in array 20. Column lines 32 may be used for readingout image signals from pixels 22 and for supplying bias signals (e.g.,bias currents or bias voltages) to pixels 22. If desired, during pixelreadout operations, a pixel row in array 20 may be selected using rowcontrol circuitry 26 and an image signal generated by each correspondingimage pixel 22 in that pixel row can be read out along a respectivecolumn line 32.

Column readout circuitry 28 may receive image signals (e.g., analogpixel values generated by pixels 22) over column lines 32. Columnreadout circuitry 28 may include memory circuitry for temporarilystoring calibration signals (e.g., reset level signals, reference levelsignals) and/or image signals (e.g., image level signals) read out fromarray 20, amplifier circuitry or a multiplier circuit, analog to digitalconversion (ADC) circuitry, bias circuitry, latch circuitry forselectively enabling or disabling the column circuitry, or othercircuitry that is coupled to one or more columns of pixels in array 20for operating pixels 22 and for reading out image signals from pixels22. ADC circuitry in readout circuitry 28 may convert analog pixelvalues received from array 20 into corresponding digital pixel values(sometimes referred to as digital image data or digital pixel data).Column readout circuitry 28 may supply digital pixel data to control andprocessing circuitry 24 and/or processor 18 (FIG. 1) for pixels in oneor more pixel columns.

If desired, pixel array 20 may also be provided with a filter arrayhaving multiple (color) filter elements (each corresponding to arespective pixel) which allows a single image sensor to sample light ofdifferent colors or sets of wavelengths. In general, filter elements ofany desired color and/or wavelength (e.g., infrared wavelengths) and inany desired pattern may be formed over any desired number of imagepixels 22. In the illustrative example of time-of-flight sensing usingan illumination source, pixel array 20 may be provided with a correspondfilter array that passes light having colors and/or frequencies emittedfrom the illumination source.

Sensor module 16 may include one or more arrays 20 of image pixels 22.Image pixels 22 may be formed in a semiconductor substrate usingcomplementary metal-oxide-semiconductor (CMOS) technology orcharge-coupled device (CCD) technology or any other suitablephotosensitive devices technology. Image pixels 22 may be frontsideillumination (FSI) image pixels or backside illumination (BSI) imagepixels. If desired, array 20 may include pixels 22 of different typessuch as active pixels, optically shielded pixels, reference pixels, etc.If desired, sensor module 16 may include an integrated circuit packageor other structure in which multiple integrated circuit substrate layersor chips are vertically stacked with respect to each other.

Configurations in which TOF sensor 12 (FIG. 1) is configured to performindirect TOF measurements based on phase differences between a modulatedlight signal (emitted by illumination module 14 in FIG. 1) and thereflected modulated light signal from an object in an image scene(received by sensor module 16 in FIG. 2) are described herein forillustrative purposes. In these configurations, sensor module 16 mayinclude an array of “lock-in” active pixels 22, each configured todemodulate the received light signal based on a sensor modulationfrequency to generate corresponding charge portions useable to generateTOF information.

FIG. 3 is a circuit diagram of an illustrative image sensor pixel 22(e.g. implementing each lock-in pixel 22 forming array 20 in FIG. 2).Pixel 22 may include photosensitive element 40 (e.g., photodiode 40).Photodiode 40 may receive incident light over a period of time (e.g., anintegration time period) and may generate electric charge based on theincident light. A first terminal of photodiode 40 may be coupled to avoltage terminal 38 such as a ground voltage source. An anti-bloomingtransistor 42 may couple photodiode 40 (e.g., a second terminal ofphotodiode 40) to a voltage terminal 44 such as a supply voltage source.Transistor 42 may be configured to prevent blooming at photodiode 40and/or may serve to keep photodiode 40 at a reset voltage level (e.g.,the supply voltage level). As an example, when control signal AB isasserted (e.g., at a logic high to turn on transistor 42), photodiode 40may be reset to the supply voltage level. When control signal AB isdeasserted (e.g., at a logic low to turn off transistor 42), photodiode40 may begin to accumulate charge in response to incident light.

Pixel 22 may include (local) charge storage regions such as storagegates 46 and 56. As an example, each storage gate may include acorresponding adjustable charge transfer barrier portion and acorresponding charge storage portion over which the gate terminal isformed. In other words, control signals SG1 and SG2 may be adjusted tocontrol the flow of charge from photodiode 40 into the charge storageregions associated with storage gates 46 and 56, respectively. The useof storage gates in pixel 22 is merely illustrative. If desired, anysuitable types of charge storage regions may be used in pixel 22.

Transistors 45 and 55 may couple photodiode 40 to storage gates 46 and56, respectively. Control signals MOD1 and MOD2 may be used to activetransistors 45 and 55, respectively, and may be used to selectivelytransfer charge generated by photodiode 40 to one of storage gates 46 or56 during the integration time period. As an example, control signalsMOD1 and MOD2 may be inverted versions of each other during theintegration time period. As such, at most only one of transistors 45 or55 may be activated at a given time, thereby separating image chargegenerated at photodiode 40 into first and second charge portions storedat storage gates 46 and 56, respectively, depending on the time periodsduring which respective signals MOD1 and MOD2 are asserted (e.g.,depending on a sensor modulation signal based on which pixel 22 orsensor module 16 is modulated).

Pixel 22 may include floating diffusion region 60 having an associatedcharge storage capacity (e.g., having capacitance C_(FD) relative tovoltage terminal 50). As an example, floating diffusion region 60 may beimplemented as a doped semiconductor region (e.g., a region in a siliconsubstrate that is doped by ion plantation, impurity diffusion, or otherdoping processes). Storage gates 46 and 56 may temporarily store(portions of) image charge generated at photodiode 40 prior totransferring the stored portions of image charge to floating diffusionregion 60 for readout.

Transfer transistors 48 and 58 may respectively couple storage gates 46and 56 to floating diffusion region 60. During readout operations, eachtransfer transistor (when activated by control signals TX1 or TX2) maytransfer a charge portion stored at the corresponding storage gate tofloating diffusion region 60 for readout. A reset transistor 62 maycouple floating diffusion region 60 to a voltage terminal 52 such as asupply voltage source. As an example, when control signal RST isasserted, floating diffusion region 60 may be reset to a reset voltagelevel (e.g., the supply voltage level). If desired, transistor 62 (incombination with other transistors) may be used to reset other portionsof pixel 22 (e.g., storage gates 46 and 56 using transistors 48 and 58)to the reset voltage level.

Pixel 22 may include source follower transistor 64 and row selecttransistor 66 (sometimes collectively referred to herein as pixelreadout circuitry). Source follower transistor 64 has a gate terminalcoupled to floating diffusion region 60, a first source-drain terminal(e.g., one of a source or drain terminal) coupled to voltage terminal 54(e.g., a supply voltage source), and a second source-drain terminal(e.g., the other one of the source or drain terminal) coupled to rowselect transistor 66. Transistor 66 may have a gate terminal that iscontrolled by row select control signal SEL. When control signal SEL isasserted (e.g., during a pixel readout operation when reset and/or imagelevel signals from one or more pixels are being read out), a pixeloutput signal may be passed onto path 70 (e.g., column line 32 in FIG.2). The pixel output signal may be an output signal having a magnitudethat is proportional to the amount of charge at floating diffusionregion 60.

The configuration of pixel 22 shown in FIG. 3 is merely illustrative. Ifdesired, pixel 22 in FIG. 3 may include one or more suitable additionalelements (e.g., elements analogous to transistors 45 and 48, and storagegate 46 along one or more additional parallel paths between photodiode40 and floating diffusion region 60, etc.), may exclude one or moresuitable elements, and/or may replace one or more suitable elements(e.g., replace storage gates 46 and 56 with other types of chargestorage structures, etc.). If desired, any of the voltage terminals inpixel 22 may be coupled to a variable voltage source or a fixed voltagesource.

Configurations in which an image sensor pixel array such as array 20 inFIG. 2 includes pixels 22 each having the implementation of pixel 22shown in FIG. 3 are described herein as illustrative examples. Ifdesired, the embodiments described herein may similarly apply to anarray having pixels of other implementations. In general, any suitableconfiguration for a lock-in pixel or any other suitable types of(indirect) TOF sensing pixels may be used.

FIG. 4 is an illustrative diagram showing how illumination module 14 mayemit light and how sensor module 16 may receive the correspondingreflected light (e.g., the emitted light after reflecting off of one ormore objects). As shown in FIG. 4, illumination module 14 may include anemitter 80 coupled to driver circuitry 82 (sometimes referred to ascontroller circuitry 82) for emitter 80. Emitter 80 (sometimes referredto herein as a light source, a light emitter) may be implemented usingand may include one or more light emitting diodes (LEDs), one or morelaser diodes, one or more lasers, and/or one or more of other suitablelight or illumination sources. Emitter 80 may be controlled by one ormore control, clock, and/or modulation signals from driver circuitry 82.As an example, one or more (modulation) signals from driver circuitry 82may control emitter 80 to emit light having a sinusoidal pattern orwaveform and to exhibit the desired illumination characteristics such asa desirable modulation frequency, a desirable modulated amplitude, etc.As another example, one or more (control and clock) signals from drivercircuitry 82 may control emitter 80 to periodically emit short pulses oflight and to exhibit the desired illumination characteristics such as adesired peak power for each of the pulses, a suitable time periodbetween adjacent pulses or a suitable pulse frequency, etc.

Controlled by driver circuitry 82, emitter 80 may emit light 90 havingany suitable characteristics (e.g., any suitable waveform, any suitablepeak amplitude or power, any suitable periodicity or frequency, etc.).Light 90 may reach one or more objects 86 in an image scene and reflectoff one or more objects 86 as reflected light 100. Objects 86 mayinclude any suitable objects, inanimate or animate.

Reflected light 100 may be received at one or more (active) pixels 22 insensor module 16. Sensor module 16 may also include driver circuitry 84(e.g., in row control circuitry 26 in FIG. 2, in control and processingcircuitry 24 in FIG. 2, etc.) coupled to pixels 22. As an example,driver circuitry 84 may provide control signals to control theoperations of pixels 22 (e.g., provide corresponding control signalscoupled to transistors or other actuated elements in pixels 22). Inparticular, based on the received control signals from driver circuitry84, pixels 22 may generate different portions of charge in response toreflected light 100 (e.g., during an integration time period), mayperform readout operations on the generated portions of charge (e.g.,during a readout time period), and may perform other suitable operationsduring other time periods.

In some configurations where illumination module 14 and sensor module 16operate in an indirect TOF sensing scheme using lock-in pixels 22,driver circuitry 82 may provide a modulation signal to emitter 80 toemit light signal 90 having a given modulation frequency, and reflectedlight 100 may have the same modulation frequency with a phase delaycorresponding to the distance of object 86 off of which light signal 100is reflected. In these configurations, sensor module 16 may be operatedbased on the given modulation frequency. As an example, driver circuitry84 may receive a sensor modulation signal having the given modulationfrequency and may control pixels 22 based on the sensor modulationsignal and the given modulation frequency. Based on the sensormodulation (using the modulation signal), each pixel 22 may becontrolled to generate and store multiple portions of charge (e.g., atstorage gates 46 and 56 in FIG. 3) in response to received light signal100. Each pixel 22 may thereby be used to generate cross-correlationdata useable to generate TOF information about the scene.

The TOF sensing system in FIG. 4 is merely illustrative. Illuminationmodule 14 and sensor module 16 may each include any other suitablecircuitry (e.g., power supply circuitry, processing circuitry, controlcircuitry, readout circuitry, timing circuitry, clock circuitry, etc.).While illumination module 14 and sensor module 16 are shown ascompletely separate modules in FIG. 4 (and in FIG. 1), this is merelyillustrative. If desired, illumination module 14 and sensor module 16may be coupled to and include shared circuitry in the camera modulesystem (e.g., power supply circuitry, clock generation circuitry, atiming controller, signal generator circuitry, control circuitry,storage circuitry, etc.), and may operate in close connection with eachother.

FIG. 5 is an illustrative diagram showing different signalcharacteristics of different signals in an illustrative TOF sensingsystem (e.g., the system shown in FIG. 4). The top plot of FIG. 5 showsthe power characteristics of an illumination signal (e.g., light signal90 in FIG. 4) over time (e.g., also over distance as the illuminationsignal propagates through time). The illumination signal in FIG. 5 mayexhibit a sinusoidal waveform having a modulated amplitude and acorresponding modulation frequency. As an example, driver circuitry 82in FIG. 4 may provide a corresponding modulation signal to modulate theamplitude of the illumination signal and control the illumination toexhibit the modulation frequency, thereby controlling illuminationmodule 14 to provide the illumination signal in FIG. 5. While, in theexample of FIG. 5, the illumination signal exhibits a power between 0 Wand 6 W, this is merely illustrative.

The middle plot of FIG. 5 shows the sensor or pixel modulationcharacteristics of a sensor modulation signal (e.g., a sensor modulationsignal received by driver circuitry 84 in FIG. 4, a sensor modulemodulation signal used by driver circuitry 84 to generate controlsignals for controlling pixels 22 in FIG. 4) over time. The sensormodulation signal in FIG. 5 may exhibit a sinusoidal waveform having thesame modulation frequency as the illumination signal. As an example,driver circuitry 84 in FIG. 4 may receive the sensor modulation signaland may generate control signals for each pixel 22 based on the sensormodulation signal. In particular, the sensor modulation signal may beused to selective assert and deassert control signals MOD1 and MOD2 foreach pixel 22 (FIG. 3), thereby controlling the charge collection andstorage at different storage regions (e.g., storage gates 46 and 56 inFIG. 3) in each pixel 22. The different charge portions separated bycontrol signals MOD1 and MOD2 may separately read out and used toidentify (e.g., calculate) the corresponding phase difference betweenthe illumination signal and the reflected signal received by sensormodule, and consequently, the corresponding TOF information may also beidentified (e.g., calculated).

The bottom plot of FIG. 5 shows the pixel response correlationcharacteristics of the corresponding cross-correlation data or signal(e.g., between the illumination signal and the sensor signal generatedbased on the sensor modulation and calculated from the different signalreadout operations from each pixel 22) over time (e.g., over distancetraveled by light signals 90 and 100). In other words, based on thesensor modulation signal (e.g., controlling pixel 22 to outputcorresponding signals during different readouts) and the illuminationsignal in FIG. 5, processing circuitry for the camera module (e.g.,processing circuitry 24 in FIG. 2 or processing circuitry 18 in FIG. 1)may generate the cross-correlation data in FIG. 5. In particular, at themaximum pixel response correlation, the sensor-generated signalindicative of the reflected light may correlate best with theillumination signal. As such, based on the peaks of pixel responsecorrelation, the processing circuitry may identify and determine thephase difference between the illumination light and the reflected light,and may consequently determine TOF (e.g., distance or depth)information.

While in the example of FIG. 5 the illumination signal emitted may be amodulated sinusoidal signal, it may be desirable to provide illuminationsignals having other characteristics. In particular, it is oftenimpractical to create a harmonically-modulated illumination signal, andaccordingly, such illumination systems may not be as readily availableas illumination systems of other types. As such, an illumination modulesuch as illumination module 14 in FIG. 4 may be configured to emit light90 having other characteristics.

FIG. 6 is an illustrative diagram showing an illumination signal havingperiodic illumination pulses that may be implemented by an illustrativeTOF sensing system (e.g., the system shown in FIG. 4). In the example ofFIG. 6, the sensor modulation signal in the middle plot may remain thesame as (e.g., have the same characteristics as) the sensor modulationsignal in FIG. 5. However, as shown in FIG. 6, the illumination signalin the top plot may exhibit narrow pulses of relatively high power(compared to the peak power in the illumination signal of FIG. 5). As anexample, driver circuitry 82 (FIG. 4) may control emitter 80 may exhibitnarrow width pulses of light having a regular periodicity (e.g., havinga regular pulse frequency).

As shown in FIG. 6, due the narrow width of the pulses in theillumination signal, the illumination signal may be associated with acorresponding low or reduced duty cycle such as a duty cycle of lessthan 20%, less than 10%, less than 5%, less than 1%, or any othersuitable duty cycle. Accordingly, control or drive signals provided bydriver circuitry 82 and received by emitter 84 (FIG. 4) to exhibit theillumination signal of FIG. 6 may similarly have a corresponding low orreduced duty cycle. Additionally, the illumination signal in FIG. 6,exhibiting an increased peak power of 15 W instead of 6 W exhibited bythe illumination signal of FIG. 5, may still have the same totalillumination power as the illumination signal of FIG. 5. Advantageously,with a short enough of a pulse width for each of the pulses whilemaintaining the same total illumination power as the illumination signalin FIG. 5 (e.g., illumination signals normalized with respect to eachother), the distortion on the cross-correlation data (e.g., thedifference between the cross-correlation data in FIG. 6 compared to inFIG. 5) is minimal. In other words, the illustrative TOF sensing systemimplementing the illumination and sensor modulation characteristics ofFIG. 6 can effectively have (almost) the same cross-correlationcharacteristics (e.g., the same cross-correlation plot) as theillustrative TOF sensing system implementing the illumination and sensormodulation characteristics of FIG. 5.

While in the example of FIG. 6, the periodicity or frequency of theillumination pulses (e.g., the pulse frequency in the illuminationsignal) may be the same as that of the sensor modulation signal, this ismerely illustrative. If desired, the illumination signal may exhibitperiodic pulses that are less frequent than those shown in FIG. 6. As aparticular example, every other pulse in the illumination signal of FIG.6 may be omitted. In these scenarios where one or more pulses areomitted, the peak power may be increased accordingly to keep the totalillumination power the same and maintain normalization (e.g., if everyother pulse is omitted, the peak power of the remain pulses may beapproximately doubled). By keeping the total illumination powerconsistent, maintaining narrow pulse widths, and using the same sensormodulation signal as in FIGS. 5 and 6, each of these illustrative TOFsensing system (e.g., implementing an illumination pulse frequency lessthan a sensor modulation frequency) can maintain the samecross-correlation characteristics (as shown in FIGS. 5 and 6).

In the most extreme case, the periodic pulses in the illumination signalof FIG. 6 for a single TOF sensing operation may be replaced by a singlestrong pulse for the single TOF sensing operation (e.g., every pulse inthe illumination signal shown in FIG. 6 may be omitted except one). FIG.7 is an illustrative diagram showing an illumination signal exhibiting asingle pulse for each TOF sensing operation that may be implemented byan illustrative TOF sensing system (e.g., the system shown in FIG. 4).In the example of FIG. 7, the sensor modulation signal in the middleplot may remain the same as (e.g., have the same characteristics as) thesensor modulation signals in FIGS. 5 and 6. To keep total illuminationpower consistent, the single pulse in the illumination signal of FIG. 7may exhibit a relatively large peak power (e.g., 150 W compared to 15 Win the example of FIG. 6). This may help in keeping thecross-correlation characteristics in the bottom plot of FIG. 7 almost(or effectively) identical to the cross-correlation characteristics inthe bottom plots of FIGS. 5 and 6.

As an illustrative example, the system in FIG. 4 may use (or exhibit)signals having characteristics described in connection with FIG. 7. Inparticular, driver circuitry 82 may control emitter 80 to generate andemit an illumination signal having a single illumination pulse (e.g.,having a large peak power and a relatively low pulse width or dutycycle) for each TOF sensing operation (involving any number of suitablereadout operations of corresponding charge portions at each pixel 22 togenerate one set of corresponding TOF information about the scene).Driver circuitry 84 may receive a sensor modulation signal having amodulation frequency and may generate control signals for pixels 22based on the sensor modulation signal.

As an example, driver circuitry 84 may assert and deassert controlsignals MOD1 and MOD2 in FIG. 3 based on the sensor modulation signal orthe method frequency to split photodiode-generated charge betweenstorage gates 46 and 56 in FIG. 3. Based on the different chargeportions, each pixel 22 may generate a set of image signals(corresponding to the different charge portions collected duringdifferent phases of the modulation signal). Downstream readout circuitryand/or processing circuitry (e.g., readout circuitry 28 and processingcircuitry 24 in FIG. 2) may be coupled to pixels 22 in FIG. 4 maydetermine the cross-correction data in the bottom plot of FIG. 7 usingthe image signals output by pixels 22. Accordingly, the processingcircuitry may also identify a phase difference and consequently thedepth or distance of a target object (and similar properties foradditional objects) based on the cross-correlation data.

While keeping the cross-correlation characteristics effectively the same(relative to scenarios in which the different illumination signals ofFIGS. 5 and 6 are used), a TOF sensing system employing signals havingcharacteristics shown in FIG. 7 may exhibit a number of advantages. Asmentioned above, illumination modules that produce narrow and strongpulses (e.g., the pulse in the illumination signal of FIG. 7) at lower(pulse) frequencies, and at lower duty cycles are more readily availablethan illumination modules that can operate with high modulationfrequencies and that exhibit perfect sinusoidal behavior. Consequently,the illustrative TOF sensing system employing the illumination signalconfiguration of FIG. 7 can be more readily constructed and can behaverelatively predictably.

Furthermore, the illustrative TOF sensing system employing theillumination signal configuration of FIG. 7 may help suppress or reducealiasing issues (relative to those that employ the configurations ofFIG. 5 or 6). In particular, due to the harmonic or periodic nature ofthe modulated light being emitted by the illumination module with anygiven TOF sensing operation (e.g., in the configurations of FIG. 5 or6), each calculated phase difference can represent a set of differentdepths or distances, each a full wavelength apart. While the use ofanother lower frequency modulated light signal may help reduce theseissues. At some point, the use of the lower frequency modulated lightsignal can also be aliased.

Take as an illustrative example the configuration of FIG. 6, which canillustrate this aliasing issues. As shown in FIG. 6, the illuminationsignal includes a series of pulses (e.g., a first pulse, a second pulse,. . . ). Because the pulses occur in a periodic manner, each point inthe cross-correlation data in FIG. 6 may be correlated with (e.g., mayoriginate from) any number of the different pulses prior to it (eachindicative of a different distance traveled by that pulse). In such amanner, a highly reflective target object at a long distance maycontribute significantly to the pixel response and may be interpreted asa target object that is nearby.

Advantageously, the illustrative TOF sensing system employing theillumination signal configuration of FIG. 7 resolves the above-mentionedambiguity contributing to aliasing by providing only a single pulse foreach TOF sensing operation. Additionally, the illustrative TOF sensingsystem may suppress the sensor modulation signal (e.g., suppress or stopsensor modulation) after a given time period (e.g., beyond a distance ofinterest traveled by the illumination pulse), thereby removing aliasingfrom the objects beyond the distance of interest. In other words, eachpoint in the corresponding generated cross-correlation data maycorrelate with (e.g., may originate from) only the single illuminationpulse and based on objects within the distance of interest. This helpsto prevent aliasing issues resulting from reflected light from objectsbeyond the distance of interest.

Moreover, the illustrative TOF sensing system employing the illuminationsignal configuration of FIG. 7 (e.g., having a reduced duty cycle and anincreased peak power associated with a single illumination pulse) mayalso improve ambient light resistance or ambient light rejection. Inparticular, as described above, by using the single illumination pulseconfiguration of FIG. 7, the illustrative TOF sensing system may alsoselectively suppress the sensor modulation for light returning fromdistances beyond the distance of interest (e.g., set a shortenedintegration time period for TOF sensing). Doing so not only allows theillustrative TOF sensing system to reduce aliasing effects from longerdistances but also allows the sensor to reduce the amount of overallincident ambient light integrated at pixels 22.

While some systems may remove ambient light from the pixel after theambient light has been integrated by pixels (e.g., by using some controlor reference signal subtraction after pixel readout), these systemscannot also remove the corresponding ambient light photon shot noise inthe already-integrated ambient light charge. As such, the illustrativeTOF sensing system employing the illumination signal configuration ofFIG. 7 and selectively suppressing sensor modulation is moreadvantageous than these systems (e.g., employing reference lightsubtraction) by integrating less overall ambient light charge andinherently reducing the corresponding ambient light photon shot noise.

As an illustrative example, FIG. 8 shows a sensor modulation signal thatis selectively suppressed when used in combination with an illuminationsignal exhibiting a single pulse (as described above in connection withFIG. 7). The illumination signal of FIG. 8 may have the samecharacteristics as the illumination signal of FIG. 7 (e.g., employing asingle strong narrow width pulse for each TOF sensing operation). Asshown in FIG. 8, the sensor modulation signal in the middle plot maybegin with the pulse of the illumination signal in the top plot.However, after three cycles (e.g., corresponding to a distance ofinterest or a desired time for light emission and reflection), thesensor modulation signal may be suppressed or stopped.

In other words, (based on the examples described in connection withFIGS. 3 and 4) during the three cycles of the sensor modulation signal,driver circuitry 84 receiving the sensor modulation signal may generatecontrol signals MOD1 and MOD2 (e.g., alternatingly assert controlsignals MOD1 and MOD2 periodically) in pixel 22 to store and integratethe corresponding charge portions. After the three cycles of the sensormodulation signal, driver circuitry 84 may stop generating (e.g.,deassert) both control signals MOD1 and MOD2 that were previouslygenerated based on the sensor modulation signal. Consequently, nofurther charge is added to the corresponding charge portions (e.g.,stored at storage regions 46 and 56 in pixel 22 in FIG. 3). Accordingly,the cross-correlation data in FIG. 8 shows the effects the correspondingsensor modulation suppression on the pixel response. The illustrativeTOF sensing system employing the illumination signal configuration andselective sensor modulation suppression of FIG. 8 may provide aliasingrejection and ambient light rejection.

Various embodiments have been described illustrating systems and methodsfor indirect time-of-flight (TOF) sensing using a system employingreduced duty cycle illumination.

As an example, a time-of-flight sensing system may include anillumination source, pixels, driver circuitry (e.g., in row controlcircuitry or pixel control circuitry) coupled to the pixels, andprocessing circuitry.

The illumination source may be configured to emit an illumination signal(e.g., a light signal) having a plurality of pulses each for acorresponding time-of-flight sensing operation. If desired, theillumination signal may have a duty cycle of less than 20 percent. Ifdesired, the plurality of pulses may be periodic, and the illuminationsignal may have a pulse frequency associated with the periodicity.

The pixels may be configured to receive reflected light resulting froman emitted light pulse and to generate charge in response to thereceived light during a time-of-flight sensing operation. If desired,each of the pixels may include a photosensitive element coupled to firstand second charge storage region via first and second transistors,respectively. The first and second transistors for the given pixel maybe coupled to a floating diffusion region and pixel readout circuitry(e.g., a source follower transistor and a row select transistor).

The driver circuitry may be configured to control the pixels based on asensor modulation signal having a modulation frequency during thetime-of-flight sensing operation. The pulse frequency may be less thanthe modulation frequency. If desired, the driver circuitry may providefirst and second corresponding control signals to the first and secondtransistors for at least a given pixel based on the sensor modulationsignal. The first and second transistors for the given pixel may bealternatively activated by the first and second corresponding controlsignals to store the generated charge in first and second portions atthe first and second charge storage regions, respectively, during thetime-of-flight sensing operation. The given pixel may be configured toperform separate readout operations for each of the first and secondportions of the generated charge.

The processing circuitry may be configured to generate cross-correlationdata based on the generated charge at the pixels to identifytime-of-flight information (e.g., depth or distance information for oneor more objects in a scene).

If desired, the sensor modulation signal may be used to control thepixels during a first time period in the time-of-flight operation, andpixel modulation may be suppressed during a second time period in thetime-of-flight sensing operation. The pixels may be modulated to storedifferent portions of the generated charge at corresponding chargestorage regions in each pixel during the first time period, and pixelmodulation may be suppressed by stopping an integration of additionalcharge at the corresponding charge storage regions in each pixel duringthe second time period. The first time period may be associated withdistances within a target distance of interest, and the second timeperiod may be associated with distances outside of the target distanceof interest.

As another example, a method for indirect time-of-flight sensingoperations may include: emitting a light signal exhibiting light pulsesat a pulse frequency, receiving a reflected light signal resulting fromthe emitted light signal at sensor module, performing sensor modulationfor the sensor module using a modulation frequency to generate sensordata based on the reflected light signal, wherein the pulse frequency isless than the modulation frequency, after performing the sensormodulation for a time period associated with a distance of interest,stopping the sensor modulation, and determining time-of-flightinformation based on cross-correlation data obtained from the sensordata.

If desired, performing the sensor modulation may include, for eachactive pixel in the sensor module, storing different portions of chargeat corresponding charge storage regions in each active pixel andseparately reading out each of the different portions of charge fromeach active pixel. If desired, stopping the sensor modulation comprises,for each active pixel in the sensor module, stopping the integration ofadditional charge at the corresponding charge regions in each activepixel.

As yet another example, an indirect time-of-flight sensor may include:an illumination module configured to emit a light signal having oneillumination pulse for each time-of-flight sensing operation, and asensor module configured to receive a reflected light signal based on areflected version of the emitted light signal, to perform signalmodulation based on a modulation frequency to generate charge inresponse to the received light signal for a given time-of-flight sensingoperation, and to suppress sensor modulation for the given time-offlight sensing operation based on a distance of interest within whichtime-of-flight information is determined. The sensor module may beconfigured to perform ambient light rejection and aliasing rejectionwhen suppressing the sensor modulation.

The foregoing is merely illustrative of the principles of this inventionand various modifications can be made by those skilled in the artwithout departing from the scope and spirit of the invention. Theforegoing embodiments may be implemented individually or in anycombination.

What is claimed is:
 1. A time-of-flight sensing system comprising: anillumination source configured to emit a light pulse for atime-of-flight sensing operation; pixels configured to receive reflectedlight resulting from the emitted light pulse and to generate charge inresponse to the received light during the time-of-flight sensingoperation; driver circuitry configured to control the pixels based on asensor modulation signal having a modulation frequency during thetime-of-flight sensing operation; and processing circuitry configured togenerate cross-correlation data based on the generated charge toidentify time-of-flight information.
 2. The time-of-flight sensingsystem defined in claim 1, wherein each of the pixels includes aphotosensitive element coupled to first and second charge storage regionvia first and second transistors, respectively.
 3. The time-of-flightsensing system defined in claim 2, wherein the driver circuitry providesfirst and second corresponding control signals to the first and secondtransistors for at least a given pixel based on the sensor modulationsignal.
 4. The time-of-flight sensing system defined in claim 3, whereinthe first and second transistors for the given pixel are alternativelyactivated by the first and second corresponding control signals to storethe generated charge in first and second portions at the first andsecond charge storage regions, respectively, during the time-of-flightsensing operation.
 5. The time-of-flight sensing system defined in claim4, wherein the first and second transistors for the given pixel arecoupled to a floating diffusion region and pixel readout circuitry, andwherein the given pixel is configured to perform separate readoutoperations for each of the first and second portions of the generatedcharge.
 6. The time-of-flight sensing system defined in claim 1, whereinthe illumination source is configured to emit additional light pulses,and the light pulse and the additional pulses occur at a frequency lessthan the modulation frequency.
 7. The time-of-flight sensing systemdefined in claim 6, wherein the light pulse and the additional lightpulses for an illumination signal having a duty cycle of less than 20percent.
 8. The time-of-flight sensing system defined in claim 6,wherein each of the additional light pulses is associated with acorresponding additional time-of-flight sensing operation.
 9. Thetime-of-flight sensing system defined in claim 1, wherein the sensormodulation signal is used to control the pixels during a first timeperiod in the time-of-flight operation and pixel modulation issuppressed during a second time period in the time-of-flight sensingoperation.
 10. The time-of-flight sensing system defined in claim 9,wherein the pixels are modulated to store different portions of thegenerated charge at corresponding charge storage regions in each pixelduring the first time period, and wherein pixel modulation is suppressedby stopping an integration of additional charge at the correspondingcharge storage regions in each pixel during the second time period. 11.The time-of-flight sensing system defined in claim 9, wherein the firsttime period is associated with distances within a target distance ofinterest, and the second time period is associated with distancesoutside of the target distance of interest.
 12. A method for indirecttime-of-flight sensing operations comprising: emitting a light signalexhibiting light pulses at a pulse frequency; receiving a reflectedlight signal resulting from the emitted light signal at sensor module;performing sensor modulation for the sensor module using a modulationfrequency to generate sensor data based on the reflected light signal,wherein the pulse frequency is less than the modulation frequency; anddetermining time-of-flight information based on cross-correlation dataobtained from the sensor data.
 13. The method defined in claim 12,wherein performing the sensor modulation comprises, for each activepixel in the sensor module, storing different portions of charge atcorresponding charge storage regions in each active pixel and separatelyreading out each of the different portions of charge from each activepixel.
 14. The method defined in claim 13 further comprising: afterperforming the sensor modulation for a time period associated with adistance of interest, stopping the sensor modulation.
 15. The methoddefined in claim 14, wherein stopping the sensor modulation comprises,for each active pixel in the sensor module, stopping the integration ofadditional charge at the corresponding charge regions in each activepixel.
 16. The method defined in claim 12, wherein each light pulse inthe light signal is associated with a corresponding time-of-flightsensing operation based on which corresponding time-of-flightinformation is determined.
 17. The method defined in claim 12, whereinthe light signal has a duty cycle of less than 20 percent.
 18. Anindirect time-of-flight sensor comprising: an illumination moduleconfigured to emit a light signal having one illumination pulse for eachtime-of-flight sensing operation; and a sensor module configured toreceive a reflected light signal based on a reflected version of theemitted light signal, to perform signal modulation based on a modulationfrequency to generate charge in response to the received light signalfor a given time-of-flight sensing operation, and to suppress sensormodulation for the given time-of flight sensing operation based on adistance of interest within which time-of-flight information isdetermined.
 19. The indirect time-of-flight sensor defined in claim 18,wherein the light signal has a pulse frequency that is less than themodulation frequency.
 20. The indirect time-of-flight sensor defined inclaim 18, wherein the sensor module is configured to perform ambientlight rejection and aliasing rejection when suppressing the sensormodulation.